UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation

With the rapid increase of multimedia data, a large body literature has emerged to work on multimodal summarization, majority which target at refining salient information from textual and image modalities output pictorial summary with most relevant images. Existing methods mostly focus either extractive or abstractive summarization rely presence quality captions build references. We are first propose Unified framework for Multimodal Summarization grounding BART, UniMS, that integrates objectives, as well selecting output. Specially, we adopt knowledge distillation vision-language pretrained model improve selection, avoids any requirement existence captions. Besides, introduce visual guided decoder better integrate in guiding text generation. Results show our best achieves new state-of-the-art result large-scale benchmark dataset. The newly involved objective technique proven bring noticeable improvement task.